Methods of recording voice signals in a mobile set

ABSTRACT

The present invention relates to a mobile set integrating a memory efficient data storage system for the real time recording of voice conversations, data transmission and the like. The data recorder has the capacity to selectively choose the most relevant time frames of a conversation for recording, while discarding time frames that only occupy additional space in memory without holding any conversational data. The invention executes a series of logic steps on each signal including a voice activity detector step, frame comparison step, and sequential recording step. A mobile set having a modified architecture for performing the methods of the present invention is also disclosed.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a divisional of U.S. patent application Ser. No.09/747,392, filed Dec. 22, 2000.

FIELD OF THE INVENTION

The present invention relates generally to telecommunications systemsand methods for recording information during phone calls, andspecifically to providing methods for recording information to a memoryelement during a call, in which the conversation is recorded in a way toreduce the storage space utilized in the memory element.

BACKGROUND OF THE INVENTION

References for IDS Inclusion:

Re. 34,976 (cell phone digital recorder/live conversations)

U.S. Pat. No. 6,064,792 (Fox) deferred recording.

U.S. Pat. No. 4,495,647 (Burke) Digital voice storage mobile.

U.S. Pat. No. 5,519,684 (Iizuka) Digital recorder, multiple tracks.

U.S. Pat. No. 5,995,824 (Whitfield) cell phone vox.

U.S. Pat. No. 5,867,793 (Davis) cell phone vox.

U.S. Pat. No. 3,936,610 (Schiffman) switching controller—output.

Mobile phones (Cellular Phones) have become a standard form ofcommunication in industrialized countries. Communications with people inlocal and wide area cell networks is common place. An artifact of thisform of modern communications is many times it is difficult to hear thevoice of a person over a mobile phone. This difficulty stems from bothtechnological and environmental short comings inhereint in thecommunication type.

A mobile phone network is an intricate and complex array of devices. Foreasy reference this disclosure describes a GSM (Global System for MobileCommunication) style digital mobile phone system. However the inventionherein is not limited particularly to this type of system. Generally aGSM is composed of a number of Mobile Service Centers (MSC) and anintegrated Visitor Location Register (VLR) therein. The MSC/VLR areasinclude a number of Location Areas (LA) which are defined as part of agiven MSC/VLR area. Mobile sets (MS), or mobile phone subscribers, mayfreely roam within the coverage area without having to send updateinformation to the MSC/VLR area that controls the LA. The cellularnetwork is composed of all these elements and a multitude ofsubscribers, each having a mobile set (MS).

The wide spread use of mobile phones has produced a variety of differentcellular networks. Cellular networks within the same region may operateon a different technology base. Some networks experience technicaldifficulties in the transmission and reception of signal from the MS tothe Base Station (BS), the cellular networks reception area forreceiving and transmitting information to each MS. These technicaldifficulties include interference from any number of signal producingsources (including other subscribers), geographical interference,structural interference and the like. The various source canindividually or in combination contribute to poor reception of signalfrom the BS to MS, or for some error in signal from one station to thereceiving station (MS/BS) causing the transmission to be garbled ordifficult to interpret.

Additionally, the environment where mobile sets are often used ininclude places where a subscriber may not be able to dedicate their fullattention to the conversation on a cell phone (e.g. when driving anautomobile) or when the local area the subscriber is in makes hearingdifficult (as in an areas having a lot of background noise). To assistin this problem there have been numerous recent developments to allow asubscriber to record information either during or after a conversation,using their MS as either a note pad, dictation device, or recorder forconversations. In all cases the use of the cellular phone as a datastorage device produces undesired drawbacks and stretches thelimitations of existing cellular phone technology. Some cell phones dooffer a voice recording feature, either for live conversation or as adictation machine. These recorders use existing technology andessentially combine two devices into one casing, instead of integratinga data recording system into the mobile set so that the available realestate and power of a mobile set are optimized.

The signal processing and data handling of a GSM phone utilizestechnology which can be adapted for use with the present invention.Conventional GSM mobile phones possess an analog to digital signalconverter (ADC) audio filter that converts analog microphone signal todigital speech samples at a sample rate of 8 KHz with 13 bits persample. Voice encoders may process speech samples in 20 millisecondsegments, where each segment is compressed into a speech frame of Nbits. The actual number of bits per speech frame depends on theparticular speech encoder used. The speech encoder may provide for halfrate speech, full rate, enhanced rate or variable rates for adaptivemulti-rate speech. The encoder compresses speech so that the number ofbits per second is minimized while still giving good quality speech.Voice encoder frames are interleaved and coded for error correction anddetection and then transmitted to the base station.

Downlink voice operations, received through the base station, go throughthe inverse process of the voice encoder. A digital to analog (DAC)audio filter performs inverse operations of the ADC/audio filter inprocessing downlink speech frames. A voice activity detector (VAD)generates a binary flag (value 0 or 1) indicating whether the subscriberis speaking (value 0) or not (value 1). The VAD used in the GSM standardsuppresses transmission during uplink, producing speech silenceintervals to conserve the battery charge.

It is possible to utilize much of the existing signal processing of aGSM compatible phone to enhance the data storage capacity of a mobileset, and record conversations.

It is an object of the present invention to provide a mobile set havingsufficient memory capacity to store voice conversations.

Another objective of the present invention is to provide a means forallowing a subscriber to record a voice conversation in real time forlater retrieval.

Another objective of the present invention is to provide a subscriberwith the ability to record a conversation and recall the informationsomewhat contemporaneously in the same phone conversation.

Another object of the present invention is to provide a means tostreamline the manner in which voice information is recorded, makinggreater effective use of the memory element within a subscriber's mobileset.

It is still a further objective of the present invention to allowsubscribers to record both voice and data information into a mobile setmemory element, and to provide accurate time indexing so the messagescan be reproduced in the same form that they were transmitted in.

It is still a further objective of the present invention to provide adata file management system of data stored on the memory element foreasy retrieval and sorting, either through the use of the MS or anotherdevice such as a desktop computer.

At least one of the present objectives is addressed in the followingdisclosure.

SUMMARY OF THE INVENTION

The present invention relates to a mobile set having a data recordersystem. The invention relates to the capture of real time voiceconversations on mobile sets (“cellular phones”), however the system canalso be used to capture multimedia signals, e-mail and datatransmissions as the technology and capabilities of wirelesscommunications continue to expand.

In one embodiment the invention is a method in a mobile set for storingvoice recordings. The method comprises controlling a voice activitydetector (VAD) to identify speech containing time frames from at leastone uplink and at least one downlink signal and recording the speechcontaining time frames from the uplink and downlink signals such thateach time frame is recorded sequentially with a time stamp for each timeframe. In this embodiment the mobile set receives two signals formingthe two sides of a phone conversation. To preserve memory space, theindividual time frames are arranged sequentially into a single data fileand written to memory.

In a second embodiment of the present invention, a method in a mobileset for determining record worthy voice time frames is described. Themethod comprising receiving a first signal in a voice activity detector,receiving a second signal in the voice activity detector, and comparingthe first signal to the second signal. The compared signals must be ofthe same time frame. The signal having the higher voice data content isselected for recording. It is also permissible to record both signals ifboth have sufficient voice data meeting a predetermined threshold.Normally only one person is speaking, thus the method of this embodimentallows the recording of only the person who is speaking. As theconversation proceeds, both people may speak, not speak, or only oneperson speaks. The method of the present embodiment compares each uplinkand downlink time frame as paired events, but separate from thepreceding time frame, and independent of the following time frame.Signals (either uplink or downlink) containing voice data are selectedfor recording. Similar to the previous embodiment, if neither person isspeaking, then the corresponding time frames will have less than thethreshold data required for recording. Those low or no data time framesare replaced with placeholders according to a data compression scheme.The placeholders are again recorded sequentially with the datacontaining time frames so the data file may be played back with a linearrepresentation of the voice time frames. In the event both people arespeaking simultaneously, the present method selects both data containingtime frames for recording, but continues to arrange the time framessequentially. Thus while two records may have the same time frame, theyare recorded one following the other into memory. On playback, if tworecorded time frames have the same time frame, they are both convertedinto analog signals and played simultaneously over the mobile setspeaker. It is important to note there are a wide variety ofcombinations for what information is recorded and not recorded. Eitherthe higher data containing frame of one channel is recorded (droppingthe lower data containing frame completely, with no place holder) torecording all frames sequentially and everything in between.

In a third embodiment a computer-readable medium containing instructionsfor controlling a mobile set processor to record multimedia signals isdescribed. The method comprises controlling a voice activity detector tocompare a plurality of voice signals having identical time stamps andarranging the voice signals such that data containing time stampsequences are sequentially placed into a single data file. The methodalso includes controlling a processor that will identify non-voicesignals containing the same time stamp as data containing voice timestamp sequences and sequentially recording the data containing voicesignals and the corresponding time stamp non-voice signals such thatboth the voice and non-voice data signals will be sequentially recordedinto a memory element as a single data file. In this embodiment theprogression of mobile sets to handle multimedia signals is accommodated.The computer readable medium contains instructions for controlling themobile set processor first as a voice activity detector, then amultimedia signal processor. Where a mobile set will be capable ofhandling multimedia signals, the voice activity detector will determinewhich time frames of speech (either uplink or downlink) contain data.The data containing time frame is then selected for recording intomemory. Sequentially the processor identifies any companion signal, suchas video signal, having the same time stamp as the voice signal timeframe to be recorded. The non-voice time frame (video frame) is therecorded with the corresponding time frame of voice data. In this waythe appropriate video sequence of uplink or downlink video is recordedwith the speaker (such as in a two way video conference call). If thevoice signal is not recorded, the video sequence similarly will not berecorded.

In a forth embodiment, a computer-readable medium containing a datastructure for stored phone conversations is described. The datastructure stores voice signals comprising a conversation list containingan entry for each of one or more phone conversations. Each entrycomprising a single string of data records wherein each data record hasa file pointer to the next record, the last record having an end of filemarker. Each record corresponds to at least one time stamp of the phoneconversation for use in restoring the data structure to a mediaunderstandable by a subscriber. The computer-readable medium containinga data structure represents the stored data on a memory element createdby any of the previous embodiments. The data structure is composed ofconversations that are stored as computer file records. Each filerepresents a single conversation and is a single string of data brokeninto records. The records contain pointers to each following record,allowing the records to be stored non-sequentially in the memory elementyet preserving the sequential nature of the conversation on playback.The last record of a file contains an end of file marker. If the fileincludes multimedia material, such as video conference information, thetime frames of video corresponding to recorded speech are also saved.When the record is played back only the corresponding speech and videosequences are played.

A fifth embodiment describes a method in a mobile set for a subscriberto select data to be stored. The method comprises the displaying of aplurality of recording modes while indicating a selection means forchoosing a recording mode. The subscriber then selects a recording modeand the mobile set provides a confirmation signal after a recording modehas been selected. While the methods of the present invention areprincipally designed to streamline the recording of speech time frameswithout recording non-speech containing time frames, a subscriber maymanually opt to have all frames of a conversation recorded, or only oneline recorded. Thus a subscriber may manually select from a commandlist. In response to a selection from the command list, the mobile setwill record all speech time frames (both data and non-data containingframes), record only the uplink signal, only the downlink signal, oronly data containing frames of either the uplink or downlink signals.Further the command list will provide the user with the option of notrecording accompanying multimedia time frames corresponding to the voicetime frames. Or, in response to a different response from the commandlist, record the multimedia time frames independently from the voicetime frames.

A sixth embodiment of the present invention describes a method in amobile set for replaying recorded conversations. The method comprisingdisplaying a line indicating a data structure of recorded conversationsand in response to selection of the displayed line, replaying a recordedconversation. The command list allowing stored data to be played backmay be available to a subscriber during a conversation, so if asubscriber wishes to replay information he may do so while he is stillin a conversation.

Finally a mobile set having a voice recording means for storing voiceconversations is disclosed. The mobile set of the present invention canrecord signals received through the mobile set and can playback at leasta portion of those signals on the mobile set. The mobile set comprisesan uplink/downlink switch for selecting speech frames from either auplink or downlink signal, at least one switching logic controller forswitching between the uplink and downlink signals, a method of fileheader generation for generating headers for recorded speech files, arecorder controlling means for configuring and controlling of a recorderoperation in one of several modes available to a subscriber and a memoryelement capable of storing the voice recordings.

BRIEF DESCRITPION OF THE DRAWINGS

FIG. 1 illustrates the relationship of mobile sets and a base station inoperation.

FIG. 2 shows the basic logic steps for sequential file recording.

FIG. 3 shows the basic logic for multiple signal processing.

FIG. 4 shows the logic for playback of recorded data.

DETAILED DESCRIPTION OF THE INVENTION

Prior to the discussion of the present invention, certain terms usedherein convey a meaning which extends beyond their ordinary meaning inthe field of the present invention. For clarification the followingdefinitions are used in this description.

“Data stream” refers to the information stored into memory and relayedfrom the processor of the mobile set to the memory element. The datastream contains a series of data records which are formatted similar toany of a variety of computer files. Each record possesses a pointer tothe next sequential record, and the last record in the file contains andend-of-file marker. “Data stream” refers to a single stored file ofinformation and may comprise any number of data records. The “datastream” is composed of compressed data containing either digital oranalog voice information, or other electronically storable information(such as video, e-mail or computer files).

“Downlink” refers to any signal received by a mobile set regardless ofsource.

“Memory” as used herein refers to any media capable of storinginformation in electronic form. Though computers and mobile phones oftenuse flash memory for storing information, the present discussionincludes either the use of persistent memory (retaining information evenif no power is supplied to the memory element) and flash memory, havingthe characteristic of not being able to store information withoutconstant power supplied to the memory element. In the discussion of thepresent invention the term “memory” is used to signify either flashmemory, or persistent memory.

“Mobile set” is used to describe any number of portable communicationsdevices, and is not restricted to the field of commercially availablecell phones. Although “mobile set” certainly includes cellular phones,it also more generally includes any GSM compatible phone, mobilecommunications phone (such as two way radio, “walkie-talkie”, satellitephones, etcetera). The use of the term “mobile set” furthermore is notrestricted to portable communications devices based strictly on speech.The use of “mobile set” in the present invention also includes portablecommunications device which, in addition to being able to send andreceive voice signals, are also capable of sending and receiving datasignals of various types (such as video (multimedia), e-mail, computerfiles and non-voice style electronic information in general).

“Playback” refers to the recovery and restoration of data (digital oranalog) into a media the subscriber can understand. It also requires thecorrect timely organization of all the data in the same sequence asoriginally received. While the data management system of the presentinvention includes the ability to receive and record several types ofdata streams, the playback feature allows the reproduction of all storeddata as well as the ability to properly assign time codes to non-voiceinformation which may be stored. The nature of the invention in severalembodiments does not permit “true” playback. That is the playback of therecorded information is not 100% restored to its original form. Indeedoften only half of the original data (or less) will be part of the dataavailable for playback. While “true” playback is possible, it is not inany way suggested nor required in the present invention.

“Streamlining” refers to the process by which a processor accesses avariety of different data time frames and connects them into a singledata stream while preserving the identity and source of each datarecord. Streamlining is a process by which multiple data types of bothvoice and non-voice information may be connected accurately into asingle data string, and recovered later without errors in reproductionof the original various signals. The processor in a mobile set performsa number of functions at various times or cycles. The processor acts asa frame comparator, determining which time frames are to be forwarded tothe data recorder. As a frame comparator the processor may substitute ordelete any particular frame. The processor also operates as the voiceactivity detector (VAD). The combined different processes cycles theprocessor engages in to create the single data stream for recording isreferred to as “Streamlining.”

“Subscriber” refers to anyone using a mobile set.

“Uplink” refers to any transmission of information from the mobile set.

The present invention relates to a system and method of recording voiceconversations in using a mobile set. The basic structure of a mobile setused as a portable communications device is loosely shown in FIG. 1. Themobile set 20 operates in a cell 22, which exists in a largercommunications network such as a Public Land Mobile Network (PLMN) 10.Within each cell 22 are base stations 24 used as receivers andtransmitters of the signals used to communicate with a mobile set 20.The various signals into and out of the base stations 24 are controlledthrough a series of controllers, registries, and routing equipment thatmakes up different parts of the PLMN 10. For the purposes of thisdisclosure, only signals to and from the mobile set 20 are considered,and the routing of information and signals through the whole of the PLMNare not discussed. Whenever a subscriber uses a mobile set 20 tocommunicate, all uplink signals are transmitted to the base station 24of the cell 22, and all incoming signals come through the base station24 through the PLMN 10. The exception to this occurs in radio phones orother communications devices designed to communicate directly with eachother without the use of a base station.

All signals received by the mobile set 20, whether from a wire linecaller, or another mobile subscriber, will be received by a mobile set20 through the base station 24. Any transmissions from the base station24 to the mobile set 20 are referred to herein as downlink signals. Anytransmissions from the mobile station 20 to the base station 24 areuplink signals. Signals transmitted between the base station 24 and themobile station 20 are generally digital signals. It is often the casethe mobile subscribers will call each other from their mobile sets 20and those uplink signals go to a base station 24, are processed throughthe PLMN 10 before being re-transmitted to the appropriate receivingmobile set 20. As the technology and options of mobile sets and basestations (and PLMNs) increases, conference calls between multiple mobilesubscribers and wire line callers will be possible. In any combinationof communications from either wireless or wire line subscribers, thepresent invention can successfully record the voice and data signals to,and from, a mobile set.

To preserve memory space, the present invention describes a method for amobile set to storing voice recordings. In its basic form, the methodcomprises controlling a voice activity detector to identify speechcontaining time frames from at least one uplink and at least onedownlink signal. Once the speech containing time frames are identified,the speech containing time frames are recorded. The speech containingtime frames from the uplink and downlink signals are recordedsequentially with a time stamp for each time frame.

For the method of the present invention, a dedicated voice activitydetector may be used as part of the architecture in the mobile set 20.However it is more common in GSM compatible phones that part of thedesign of the GSM mobile set 20 allows the processor 108 to operate as avoice activity detector during certain operation cycles. Reference to acycle here does not mean a single clock cycle, but rather a series ofclock cycles which are required to execute a single function in theprocessor (such as encoding a speech frame, decoding a speech frame, orcomparing two frames, etcetera). This feature is generally related tothe uplink side for preserving battery life. Thus for the presentinvention the method may utilize the voice activity detector cycles ofthe processor 108 of a GSM phone and tie in the downlink signal into thevoice activity detector cycles as an extra series of instructions. Boththe uplink and downlink signals are paired based on their time framesand recorded as a single data stream into memory 112. The processorrecords each time frame of uplink and downlink signal alternatingbetween the two sources.

Referring now to FIG. 2, the method sorts received or downlink 102signals and uplink 104 signals in the processor 108. The processor 108may have a built in memory buffer 106, or it may be separate as shown.The processor 108 alternates between time frames of the downlink 102 anduplink 104 signals, arranging them into a single data stream forrecording into memory 112. Simultaneously the uplink 104 signal is sentto the antenna 120 for transmission, and the downlink 102 signal isconverted into a form the subscriber can understand at either thespeaker 124 or display 126.

Another method comprises receiving both uplink 104 and downlink 102signals and storing them in the processor buffer 106. The uplink 104 anddownlink 102 signals are compared to each other in the voice activitydetector cycle. The signals compared to each other must have the sametime stamp (be of the same time frame). In operation, each time framethat is processed through the voice activity detector 108 is assigned alogic value. Time frames designated as record worthy (value 1) arerecorded while those not record worthy (value 0) are dropped from thedata stream to be recorded. The dropped data frames are replaced with aplaceholder that permits the playback to accurately reproduce pauses inthe original conversation. The manner of replacing non-record worthytime frames with placeholders may be done by various data compressionmeans and is not per se an inventive aspect of the present invention. Inthis method only half the data of the conversation is recorded. Ingeneral conversation, only one person is speaking at a time. To preservememory 112 space the method of the present invention distinguishes thespeech and non-speech time frames and records only the speech containingtime frames. The non-speech containing time frames are dropped from thedata stream that is recorded. Because a placeholder is inserted into thespace of each time frame that is not recorded, the linear timerelationship between the speech containing time frames is not lost. Whenthe uplink 104 and downlink 102 signals for a particular time frame bothcontain no speech, only one placeholder need be inserted into therecording data stream. The placeholder for the non-speech containingframes will be restored to non-speech pauses when the data is recoveredfor playback. The signals to be recorded are then sent to memory 112while the buffer is cleared for the next batch of time frames.Processors generally operate at a much faster cycle time than the rateat which uplink and downlink time frames are loaded into the buffer.Thus the voice activity detector cycles can clear the buffer of storedspeech time frames without the buffer becoming full. Once the timeframes are selected for recording, they are arranged into a single datastream by the processor 108. This maximizes space as the data stream cannow be recorded as a computer file composed of records. Each record hasa record pointer showing where the next sequential record is. The lastrecord has an end of file marker. The file may contain records whichcontain both voice and non-voice data.

Thus the voice activity detector cycle looks at both the uplink anddownlink time frames and assigns them a logic value of one or zero. Thefollowing example shows the logic executed by the voice activitydetector cycle and the processor during a “frame comparator” step. Thevoice activity detector determines if the speech time frame (eitheruplink or downlink source) contains record worthy data. If so the timeframe is assigned a high logic value (1). If not the frame is given alow logic value (0). Once the speech frames for a given time areassigned values, they are returned to the buffer 106 for the nextprocessor cycle. The processor 108 then retrieves the data from thebuffer 106 and sends the high logic value frames to the data recorder110 for recording. The low logic value frames are dropped, andsubstituted with a placeholder as previously described (However tofurther conserve space in memory, the place holder for a low logicspeech frame may be omitted (except where both uplink and downlinksignals contain no data). The proper sequence of timing for the speechframes can be derived from only the high logic frames that arerecorded). The processor acts as a switching logic controller indetermining which time frame to record when sorting through the uplinkand downlink signals (or various uplink and downlink signals).

EXAMPLE 1

U₁=1 then record

U₁=0 then drop

D₁=0 then record

D₁=0 then drop.

The recorded data stream, after stream lining may look like U₁, D₁, U₂,D₂, etc. . . .

The high logic time frames are recorded into a single data stream toallow for the file record to be stored as a computer file. Depending onthe operation of the mobile set, the operation of the voice activitydetector cycle and file comparator cycles may be combined into just thevoice activity detector cycle. The added benefit is high logic signalsmay be sent directly to the data recorder without having to go back intothe memory buffer, which reduces the power consumption of the operation.

The methods described above are also executable when dealing withsignals from a non-voice source, such as video, text messaging, e-mailor other signal the mobile set is capable of receiving. As the abilitiesof mobile sets expand, and offer additional features to subscribers,such as mobile video conference calling, wireless e-mail and webbrowsing, the next generation of mobile set will have a much broaderarray of data to contend with. Memory for recoding information in amobile set will therefore be at a premium. The recording of videosignals accompanying voice signals (such as in a conference call) may beselectively handled so that only the video time sequences correspondingto record worthy voice signals are recorded. The uplink and downlinksignal paths would similarly be tracked so the voice and video of theappropriate source is maintained.

By way of example, if a mobile set is receiving two downlink signals ofvoice (D₁ and D₂), and two downlink signals of video, then only thevideo time frame corresponding to the record worthy voice time frame(when some one is speaking) will be recorded (FIG. 3). Thus the party ofthe actual phone conversation who is not speaking, is not recorded foreither voice, or video. The data received goes to the processor asevents, and the operation performed is either the voice activitydetection (VAD) or frame comparator (FC). The FC cycle executes a logicyielding in data recording of an uplink signal (U_(n)), downlink signal(D_(n)) or placeholder (P_(L)).

EXAMPLE 2

VAD/FC Operation and result Event Operation logic Record 1 VAD D₁ = 1 orD₁ = 0 n/a 2 VAD D₂ = 1 or D₂ = 0 n/a 3 FC If D₁ = 1, then V₁ = 1, thenrecord D₁ and D₁, V₁ V₁. 4 FC If D₁ = 0, then V₁ = 0, then drop D₁ andV₁. P_(L) 5 FC If D₂ = 1, then V₂ = 1, then record D₂ and D₂, V₂ V₂. 6FC If D₂ = 0, then V₂ = 0, then drop D₂ and V₂. P_(L) 7 VAD U₁ = 1 or U₁= 0 n/a 8 FC If U₁ = 1, then U₂ = 1, then record U₁ and U₁, U₂ U₂. 9 FCIf U₁ = 0, then U₂ = 0, then drop U₁ and U₂. P_(L)

In this example, the data stream which is recorded is derived in theframe comparator cycle, and may appear like D₁, V₁, P_(L), D₂, V₂,P_(L), U₁, U₂, P_(L). Alternatively, if the PL is not recorded wherethere is a frame of actual data, the data stream would look like D₁, V₁,D₂, V₂, U₁, U₂.

The execution of these methods originates from a computer-readablemedium containing instructions for controlling a mobile set processor torecord multimedia signals. The computer-readable medium comprisesinstructions for controlling a processor (VAD/FC) to compare a pluralityof voice signals having identical time stamps, and arranging the voicesignals such that data containing time stamp sequences are sequential ina single data file. The computer readable-medium also has instructionsfor controlling a processor to identify non-voice signals containing thesame time stamp as data containing voice time stamp sequences. Then thedata containing voice signals and the corresponding time stamp non-voicesignals are sequentially recorded such that both the voice and non-voicedata signals are recorded into a memory element as a single data file.However it is not necessary that computer files, such as text messagesor application data files, be recorded into memory in the same manner asvoice and video. These files would be stored in whole without anyinsertion of placeholders for actual data. An arrangement of multipledata files forms a data structure in the memory element.

The memory element then forms a computer-readable medium containing adata structure for storing voice signals. The data structure comprises aconversation list containing an entry for each of one or more phoneconversations. Each entry comprising a single string of data recordswherein each data record has a file pointer to the next record, the lastrecord having an end of file marker. Depending on how a particularmobile set is designed to store information, each record willcorresponding to one or more time frames of the phone conversation foruse in restoring the data structure to a media format understandable bya subscriber.

As described in the method of selecting signals for recording, the datamay contain voice and multi-media data, with fragments of various typesof data strung together into a single data stream. In the case of amobile set that also acts as a PDA (Personal Data Assistant) the mobileset may also have the ability to record computer files. Files receivedby the mobile set may be stored sequentially, or contain a file pointerin the last record of the file that identifies any attachments.

While the computer-readable medium of the present invention has adefault method of determining what data is “record worthy” a subscribermay alter the default method by instructing the mobile set to recordinformation in a variety of other methods. Therefore an additionalmethod in a mobile set for selecting data to be stored comprisesdisplaying a plurality of recording modes, indicating a selection meansfor choosing a recording mode; and in response to selection of thedisplaying a plurality of recording modes, a different method ofrecording is selected. In this manner a subscriber can choose to recordall time frames of both the uplink and downlink signals, or record onlythe uplink, or only downlink. Where multimedia files are concerned, thisoption permits the user to preserve memory space by ignoring multimediamaterial except voice. Or the subscriber can turn recording offcompletely.

Recalling stored information (FIG. 4) from the data structure involvesdisplaying a line indicating a data structure of recorded conversations,and in response to a selection of the displayed line, replaying arecorded conversation. In addition to the ability to recall a previouslyrecorded message, the mobile set of the present invention would allow asubscriber to review recorded conversations using a variety of speedcontrols, or segment replay controls (replaying a few seconds of voicewhere the audio is garbled or difficult to distinguish).

Another feature of the present invention is the ability to recall andplayback recorded conversations while using the mobile set as a phone.In this manner a subscriber may recall a previously recordedconversation (either of the current call, or a previous call) and playit back for the subscriber, or transmit the recorded data through theuplink signal.

Finally a mobile set having a voice recording means for storing voiceconversations is disclosed. The mobile set of the present invention canrecord signals received through the mobile set and can playback at leasta portion of those signals on the mobile set. The mobile set comprisesan uplink/downlink switch for selecting speech frames from either auplink or downlink signal, at least one switching logic controller forswitching between the uplink and downlink signals, a method of fileheader generation for generating headers for recorded speech files, arecorder controlling means for configuring and controlling of a recorderoperation in one of several modes available to a subscriber and a memoryelement capable of storing the voice recordings.

1. A method in a mobile set for selecting data to be stored, comprising:(a) displaying a plurality of recording modes, each of the plurality ofrecording modes recording a different set of data frames exchangedbetween the mobile set and a second device during a phone call; (b)choosing one of the displayed plurality of recording modes; and (c)recording a set of data frames according to the chosen recording mode,the recorded set of data frames comprising a downlink voice signal, adownlink video signal, an uplink voice signal and an uplink video signalin which the downlink video signal is recorded at the mobile set whenthe downlink voice signal is determined to have voice activity and inwhich the uplink video signal is recorded at the mobile set when theuplink voice signal is determined to have voice activity.
 2. The methodof claim 1, comprising: providing a confirmation signal after choosingthe recording mode; and time stamping frames of the downlink voicesignal and frames of the uplink voice signal so that the frames of thedownlink voice signal and the frames of the uplink voice signal can bepaired according to their time stamps and recorded as a single datastream.
 3. The method of claim 1, wherein the recorded set of dataframes comprises speech data and video data transmitted by the mobileset to the second device during the phone call, and wherein the seconddevice comprises a second mobile set.
 4. The method of claim 1, whereinthe recorded set of data frames comprises speech data and video datareceived by the mobile set from the second device during the phone call,and wherein the recorded set of data frames are connected into a singledata stream in which identity and source information is preserved foreach of the downlink voice signal, the downlink video signal, the uplinkvoice signal and the uplink video signal.
 5. The method of claim 1,wherein the recorded set of data frames comprises text messaging betweenthe mobile set and the second device during the phone call.
 6. Themethod of claim 5, comprising: time stamping frames of the downlinkvoice signal, frames of the uplink voice signal and frames of the textmessaging so that the frames of the downlink voice signal, the frames ofthe uplink voice signal and the frames of the text messaging can beassociated according to their time stamps and recorded as a single datastream.
 7. The method of claim 1, wherein the recorded set of dataframes are connected into a single data stream that is recorded suchthat identity and source information is preserved for each of thedownlink voice signal, the downlink video signal, the uplink voicesignal and the uplink video signal.
 8. A method in a mobile set forreplaying recorded conversations, comprising: (a) providing a displayindicating a data structure of recorded conversations, the recordedconversations comprising uplink data frames transmitted from the mobileset to a second device during a phone call and downlink data framestransmitted, from the second device to the mobile set during the phonecall, wherein the uplink data frames comprise uplink voice signal framesand uplink video signal frames, wherein the downlink data framescomprise downlink voice signal frames and downlink video signal frames,wherein the uplink data frames and the downlink data frames areselectively recorded based on data content analysis, performed by themobile set, of each uplink data frame and each downlink data frame,wherein the uplink video signal frames are recorded when the uplinkvoice signal frames are determined to have voice activity, wherein thedownlink video signal frames are recorded when the downlink voice signalframes are determined to have voice activity; and (b) in response toselection of the displayed line, replaying a recorded conversation. 9.The method of claim 8, wherein the recorded conversations are connectedinto a single data stream that is recorded such that identity and sourceinformation is preserved for each of the downlink voice signal frames,the downlink video signal frames, the uplink voice signal frames and theuplink video signal frames.
 10. The method of claim 8, wherein therecorded conversations comprise text messaging between the mobile setand the second device during the phone call.
 11. The method of claim 10,comprising: time stamping the downlink voice signal frames, the uplinkvoice signal frames and text messaging frames so that the downlink voicesignal frames, the uplink voice signal frames and the text messagingframes can be associated according to their time stamps and recorded asa single data stream.
 12. The method of claim 8, wherein the datacontent analysis comprises a determination of data content level. 13.The method of claim 8, wherein the data content analysis comprises adetermination of voice activity.
 14. A method for replaying previouslyrecorded conversations during a real time conversation, comprising: (a)providing a display indicating data structures representing recordedconversations, the recorded conversations comprising uplink data framestransmitted from the mobile set to a second mobile set during a phonecall and downlink data frames transmitted from the second device to themobile set during the phone call, wherein the uplink data framescomprise uplink voice signal frames and uplink video signal frames,wherein the downlink data frames comprise downlink voice signal framesand downlink video signal frames, wherein the uplink data frames and thedownlink data frames are selectively recorded based on data contentanalysis, performed by the mobile set, of each uplink data frame andeach downlink data frame, wherein the uplink video signal frames arerecorded when the uplink voice signal frames are determined to havevoice activity, wherein the downlink video signal frames are recordedwhen the downlink voice signal frames are determined to have voiceactivity; and (b) in response to selection of a particular datastructure of the data structures, replaying at least a portion of theparticular data structure.
 15. The method of claim 14, wherein thedisplay can be accessed during a real time subscriber conversation usingthe mobile set without interfering in the communication between thesubscriber and a base station.
 16. The method of claim 14, wherein inresponse to the selection of the particular data structure, a portion ofa previously recorded conversation may be played back and transmittedthrough the uplink signal.
 17. The method of claim 14, wherein therecorded conversations are connected into a single data stream that isrecorded such that identity and source information is preserved for eachof the downlink voice signal frames, the downlink video signal frames,the uplink voice signal frames and the uplink video signal frames. 18.The method of claim 14, wherein the recorded conversations comprise textmessaging between the mobile set and the second device during the phonecall.
 19. The method of claim 18, comprising: time stamping the downlinkvoice signal frames, the uplink voice signal frames and text messagingframes so that the downlink voice signal frames, the uplink voice signalframes and the text messaging frames can be associated according totheir time stamps and recorded as a single data stream.
 20. The methodof claim 14, wherein the data content analysis comprises a determinationof voice activity.